1. Field of the Invention
The present invention relates to testing technique of a language processing system which receives new language syntax. More particularly, the present invention relates to a syntax coverage percentage measuring system which is composed of a syntax coverage percentage measuring unit which is used for the measurement of a syntax coverage percentage of test data and a program generating unit provided for the syntax coverage percentage measuring unit and generating programs to realize the syntax coverage percentage measuring unit. Here, the test data is data of a test input file described in accordance to BNF (Backus Normal (Naur) Form) syntax rules, and the syntax coverage percentage is indicative of a rate of BNF syntax rules used in the test input file described in accordance to the BNF syntax to the whole BNF syntax rules.
2. Description of the Related Art
As the techniques about a conventional test data generating system and a conventional test program and data generating system, there are known, for example, a test data generation supporting apparatus disclosed in Japanese Laid Open Patent application (JP-A-Heisei 3-075840, a test program creation supporting apparatus disclosed in Japanese Laid Open Patent application (JP-A-Heisei 6-214775). Also, a language processing program verifying system is disclosed in Japanese Laid Open Patent application (JP-A-Heisei 1-220044). Below, these conventional techniques will be described.
(1) The Japanese Laid Open Patent Application (JP-A-Heisei 3-075840)
As shown in FIG. 1, in the conventional test data generating system, a syntax coverage percentage of an existing test source program used to test a language processing program shown in FIG. 1 is determined, and a source program using the syntax rules which are not used in the existing test source program is produced and replenished based on the determination result. Thus, a set of the test source programs is prepared to secure syntax coverage.
In this conventional technique, the syntax of each existing test source program is fixed, and a syntax coverage determining section measures a coverage percentage only to the predetermined syntax rules. Therefore, the syntax coverage determining section must be reformed because the test data is formed in accordance with new syntax, when the conventional language processing system is used for the new syntax.
(2) Japanese Laid Open Patent Application (JP-A-Heisei 6-214775)
As shown in FIG. 2, in the conventional test data generating system, a sentence pattern generating section automatically produces test data or a test program matching to BNF syntax rules described in a BNF file.
However, the test data matching to the BNF syntax rules exists infinitely. Therefore, even if a part of the test data is taken out, the possibility that significant test items are efficiently tested based on the test data part is low. Also, because the test data is automatically generated from the syntax rules in the conventional technique, what the generated test data is semantically related to is obscure to an operator. In other words, whether the test using the generated test data is enough is obscure to the operator. Moreover, in the conventional technique, the operator can concern the processing of the test data but the number of processes becomes enormous.
(3) Japanese Laid Open Patent Application (JP-A-Heisei 1-220044)
As shown in FIG. 3, in the conventional test program and data generating system, a TP automatic creating section inputs syntax and semantic description, and a generation rule selecting section randomly selects ones of the rules and produces a test program or a verification program.
In the conventional technique, by introducing the semantic description, significant test items can be generated. Generally, in case of the development of a language processing system, syntax rules are described such that a machine can read them but semantic description is carried out in native language in many cases. Therefore, to apply this conventional technique, the semantic description must be rewritten into a format which the machine can read. For this reason, a test process becomes necessary.
As mentioned above, the test data generating system/test program and data generating system are conventionally known.
However, when the operator is conscious of the test items to produces test data, and points out a portion of the syntax rules which is not covered by the test data, the test efficiency can be improved, compared with the conventional test data generating system/test program and data generating system. Especially, in the test for the development of a language processing system, a coverage percentage measurement based on the syntax rules is effective. The operator often sets test items for each of the syntax rules which are accepted by the language processing system as a test object. Therefore, untested test items can be easily estimated from the syntax rules which are not covered by the test data.
In the above mentioned conventional examples, when a new language is designed and the language processing system is tested, the syntax coverage percentage measuring unit or the syntax coverage percentage testing unit must be built newly, because the test data is produced in accordance with the new language syntax. Therefore, the more processes are necessary to develop a syntax coverage percentage measuring unit newly.
In conjunction with the above description, Japanese Laid Open Patent application (JP-A-Heisei 10-049378) is known. In this reference, a compiler developing apparatus converts a syntactic diagram drawn when a compiler is developed, into a BNF format syntax analysis generating system code and a lexical analysis generating system code. In this conventional technique, BNF is also used. However, the role or use purpose of the output BNF format syntax analyzing generating system codes is different from that of the present invention.
Also, a coverage percentage measuring system of software is disclosed in Japanese Laid Open Patent application (JP-A-Heisei 6-4350). In this reference, in a program developing apparatus (1), a branch ID notifying process is embedded in a coverage percentage measuring object program by a branch destination process detecting section (14), a branch ID producing section (15), a branch ID notifying process embedding section (16) and a measuring table producing section (17). Also, a coverage percentage measuring table (20) is produced. In a program executing unit (3), when the coverage percentage measuring object program is executed, a branch ID corresponding to each branch destination in the actual execution is transmitted to a coverage percentage measuring unit (5) via a transmission section (32). When receiving the branch ID, a branch ID recording section (53) or a branch result recording section (54) records the branch ID in a measuring table (57). A coverage percentage analyzing section (56) periodically refers to the measuring table (57) to analyze the coverage percentage, and the analysis result is printed or displayed on a printer or a display unit (52).